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Introduction 


Recognizing that a strong commitment to advanced technical and management 
education serves to maintain the capabilities and strength of the USAF (Van Scotter, 
1993), the Air Force Institute of Technology (AFIT) Graduate School of Engineering and 
Management (GSEM) offers a range of full-time, eighteen-month, graduate-level 
scientific, engineering and management programs to military officers below the rank of 
Colonel (0-6) and Department of Defense (DoD) civilian employees. AFIT focuses on 
the USAF, the DoD, and the civilian research and development environment, striving to 
prepare students to “serve their country to the greatest degree possible” (AFIT GSEM 
Graduate Catalog, Academic Year 2001-2002, p. 3). 

Within the GSEM, the Graduate Engineering and Environmental Management 
(GEEM) program focuses on graduating civil engineering officers and civilians that can 
“integrate engineering, science, and policy issues into a decision-making framework for 
optimum management of facility operations and environmental programs” (AFIT GSEM 
Graduate Catalog, Academic Year 2001-2002, p. 170). The USAF determines the size of 
entering GEEM classes according to the need for civil engineers and the availability of 
annual funds to finance the GEEM program. Upon graduation, GEEM students are 
assigned to civil engineering organizations across the Air Force. 

The limited number of GEEM student slots compels the AFIT faculty to 
distinguish between applicants and select those most likely to successfully satisfy the 
graduation requirements: at least thirty-six quarter-hours of classes, maintaining at least a 
3.00 (on a 4.00 scale) GGPA, and a twelve quarter-hour thesis study. To this end, the 
AFIT faculty and registrar rely on objective, quantitative admission data. 
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Applicants eligible for study in the GEEM program must have an earned 
bacealaureate degree from an aceredited eollege or university in an appropriate discipline 
- most commonly civil, mechanical, or electrical engineering — an overall undergraduate 
grade point average (UGPA) of at least 3.00, and Graduate Record Examination (GRE) 
General Test score of at least 500 on the verbal section and 600 on the quantitative 
section. In addition, the AFIT GEEM program uses the undergraduate math grade point 
average (UMGPA) as an indicator of the level of mathematical study and proficiency. 

The 3.00-standard applies to the UMGPA criterion, as well. Applicants must also have 
taken an undergraduate course in ordinary differential equations (ODE). 

The existing body of research on the ability of the admission requirements to 
predict graduate school success presents inconsistent findings. Across studies, the 
predictive validity of both the GRE General Test and the UGPA have been shown to vary 
with academic discipline. Some studies report the existence of moderating effects of 
variables like age, gender, and time between undergraduate and graduate education on 
UGPA and GRE scores. Across studies, conceptualizations of graduate school success 
also vary. For example, the Educational Testing Service (ETS) validates scores on the 
GRE General Test against first-year graduate grade point average (FYGGPA). Other 
studies rely exclusively on cumulative GGPA or a dichotomous graduated versus did not 
graduate measures of graduate school performance. Other studies use composite 
measures of graduate school success. For example, Abedi (1991) used “the GGPA, the 
total number of credit hours attempted while in graduate school, and the type and number 
of degrees earned” (p. 154) to define graduate school success. 
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To counter the variability in the findings of individual studies, the 2000-2001 
Guide to the Use of Seores. published by the ETS, encourages universities and academic 
departments to perform their own validity studies on the GRE and other admission 
requirements - i.e., UGPA. While controlling for age, gender, and the time between 
undergraduate and graduate education (TDELTA), this study seeks to determine the 
validity of the AFIT admission criteria as predictors of graduate school success, defined 
separately as: GGPA, AFIT thesis grade, FYGGPA, and Coursework GGPA. 

From 1995 to 2002, the structure, content, and duration of the GEEM program 
was fairly stable, with a fairly homogenous population of students. Thus, to both 
maintain the homogeneity of the population (Keith, 1977) and to minimize “noisy data 
problems” that Schneider and Briel (1990, p.l5) propose may produce lower correlation 
between GRE scores and graduate school success, this study focuses exclusively on 
GEEM students between the classes of 1995 through 2002. 

This study seeks to answer the following questions: 

1) Do any of the individual AFIT admission requirements - UGPA, GRE 
(verbal), GRE (quantitative), UMGPA - significantly predict a 
student’s graduate school performance beyond the control variables - 
age, gender, and time between undergraduate and graduate education? 

2) Do all of the AFIT admission requirements, together, significantly 
predict a student’s graduate school performance beyond both the 
control variables and any individual admission requirement? 
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Literature Review 


An extensive body of researeh exists on the predictive validity of UGPA and GRE 
scores, the influences of age, gender, and time (in years) differences between 
undergraduate and graduate education, as well as, the various definitions of graduate 
school success. The literature review focuses on varied admissions requirements and 
processes that predict graduate school success. Research on the UGPA, GRE, age, 
gender, and TDELTA variables is presented along with a synopsis of definitions of 
graduate school success used throughout research. Finally, the AFIT admission and 
graduation requirements are presented as predictors and measures of success, 
respectively. 

Admissions Processes 

Graduate institutions evaluate applicants for admission using any of a variety of 
admission processes, characterized by both composition and complexity. The admission 
requirements that compose an admission procedure are classified as either traditional or 
non-traditional. While both categories contain objective and subjective, quantitative and 
qualitative requirements, traditional measures are those requirements that are the most 
commonly used measures of undergraduate academic performance across universities - 
i.e., UGPA and GRE General Test score. The complexity of a procedure is classified 
relative to that of other admission processes. In this study, all admissions procedures are 
compared to a baseline process that relies strictly on unaltered, traditional variables - i.e., 
the UGPA and GRE General Test score to admit applicants. Any procedure that uses a 
combination of traditional and non-traditional admission requirements - i.e., UGPA, 

GRE General Test scores, and letters of recommendation — or some altered form of a 
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quantitative admission requirement - i.e., the last two years of the UGPA — to admit 
applicants is considered, to varying degrees, more complex than the baseline procedure. 
The composition of the admission procedure, thus, drives the complexity of the 
admission process. In the following paragraphs, both traditional and non-traditional 
admission requirements will be explored in greater depth. 

Traditional Admission Requirements. Traditional requirements include objective 
measures like the UGPA and the GRE General Test score, and subjective measures like 
letters of recommendation. Since most of the applicants to graduate school have an 
UGPA, Malone, Nelson and Nelson (2001) explain that it is a “very convenient,” (p. 4) 
although range restricted, measure for admission decisions. To help further resolve 
differences between applicants clouded by the small range of UGPAs, many graduate 
institutions concurrently rely on the GRE. According to Norcross, Hyanch, and 
Terranova (1996), eighty-one percent of graduate psychology programs require GRE 
scores of applicants. The GRE provides graduate admissions with a standardized, 
common-scale measure of an applicant’s verbal, quantitative, and analytical abilities. 

Non-Traditional Admission Requirements. The category of non-traditional 
requirements contains a greater variety of less common, less-established measures than 
the traditional category. Enright and Gitomer (1989), for example, include interviews, 
writing samples, and research plans among non-traditional measures. Howard (1968) 
documents that the University of Cincinnati’s used an applicant’s UGPA from the junior 
and senior years of study to evaluate applicants with four-year cumulative UGPAs below 
graduate program requirements. The University of Cincinnati admissions committee 
suggested that the last two years of the undergraduate experience, theoretically, provided 
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more insight into the applicant’s ability to critically solve problems than do more 
traditional requirements. Stahmer (1968) reports that Harvard, Yale, and Columbia 
implemented specially-designed summer studies programs for academically “at-risk” 
applicants - those applicants with relatively low UGPA and GRE scores — and made 
admissions decisions following the completion of this program upon such non-traditional 
requirements as “formal application, transcript of college work, statement of educational 
aims, writing samples, an interview, faculty recommendations, motivation, and desire” 
(Malone, Nelson, & Nelson, 2001:6). 

Traditional Versus Non-traditional Admission Requirements. While studies 
performed by Case and Richardson (1990); Hagedom and Nora (1996); and House, 
Gupta and Xiao (1997) imply that graduate institutions adopted non-traditional measures, 
oftentimes as supplements to traditional procedures, to improve upon the low predictive 
validity of traditional measures, research has not established that non-traditional 
measures are better predictors of graduate school success than traditional measures. For 
example, Stahmer (1968) reports large percentages of “at-risk” students - i.e., those 
applicants not meeting the stated admissions minimums — admitted to graduate study 
under Harvard, Yale, and Columbia’s non-traditional measures, graduated and pursued 
further graduate study. The success of students considered “at-risk” by traditional 
measures does not prove that the non-traditional measures used to admit them to the 
Harvard, Yale, and Columbia programs are necessarily better than the traditional 
measures that would have, most likely, rejected them. It simply shows that factors other 
than the traditional measures influence the prediction of applicant graduate school 
success. The Pennsylvania State University performed a study where students both 
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above and below the minimum quantitative admission requirements were admitted for 
graduate study. No significant difference in GGPA between the two categories of 
students was found (Penn State University, Graduate School Bulletin . 1969, cited in 
Gunne and Leslie, 1972). Malone, Nelson and Nelson (2001) reported the graduation of 
seventy-five percent of the students admitted to the University of California at Los 
Angeles (UCLA) who did not meet the GPA and GRE admission requirements. Like the 
Harvard, Yale, and Columbia study (Stahmer, 1968), the graduation of these students 
from UCLA does not necessarily mean that non-traditional measures better predicted 
graduate school success over traditional measures. 

All that can be inferred from the research is that non-traditional measures may be 
worse, just as good, or better able to predict graduate school performance. Considering 
that the composition of an admissions process drives the complexity, it follows that 
graduate institutions can evaluate applicants for admissions using any degree of 
complexity and achieve worse, equal or better results in admitting applicants. 

Engineering Management Program Selection Processes. Typically, Graduate 
Engineering Management programs rely on traditional admissions processes, requiring a 
minimum UGPA and GRE General Test score. The 2001 Peterson’s Graduate Programs 
in Engineering and Applied Sciences publishes the entrance requirements for eighty-two 
graduate schools that offer degrees in Engineering Management. The majority of 
graduate schools require a minimum UGPA (most commonly either 2.5 or 3.0) and the 
GRE (verbal) and GRE (quantitative) scores, either combined or considered separately. 

A few schools deviate from the traditional admissions process. For example, the Lamar 
University, College of Graduate Studies, College of Engineering, Department of 
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Industrial Engineering, Program in Engineering Management requires at least five years 
of related work experienee. The Kansas State University, Graduate Sehool, College of 
Engineering, Department of Industrial and Manufaeturing Systems Engineering will only 
eonsider applieants with a baehelor of seienee degree in Mathematies, Engineering, or 
Physies. Dallas Baptist University requires the GMAT in lieu of the GRE General Test. 
The AFIT model, most elosely, adheres to a more traditional proeess. 

AFIT Model for Predicting Graduate School Success 

Every graduate institution is limited in the number of students it ean admit to any 
partieular elass. Whether using traditional, non-traditional, or a eombination of both 
types of admissions requirements, the goal of graduate admissions is generally the same 
aeross institutions - to seleet those applieants with the knowledge, skills, and ability to 
perform well in a respeetive graduate program. Sinee the number of applieants often 
exceeds the number of positions available, every graduate institution is compelled to 
discriminate among all applicants and admit only a percentage of the entire applicant 
pool for graduate study. From an applicant’s admissions data, graduate admissions make 
inferences about the applicant’s potential to perform in graduate school. Those 
applicants with the greatest inferred potential for performance are admitted to the 
program before applicants with lesser potential. 

The AFIT faculty and registrar rely on a combination of traditional - UGPA, GRE 
(verbal), and GRE (quantitative) — and non-traditional variables - UMGPA - to evaluate 
applicants. In the context of the GEEM program, the AFIT seeks to admit only those 
applicants with the greatest inferred potential to successfully learn to “integrate 
engineering, science, and policy issues into a decision-making framework for optimum 
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management of faeility operations and environmental programs” (AFIT GSEM Graduate 
Catalog, Aeademie Year 2001-2002, p. 170). The UGPA demonstrates the student’s 
ability to funetion in an aeademie environment by presenting a reeord of past aeademie 
aehievement. AFIT does not give any formal eonsideration to any speeifie undergraduate 
institution. The UMGPA indieates the level of mathematieal study as well as aeademie 
aehievement in mathematies eourses. The GRE assesses basie quantitative and verbal 
skills deemed requisite for sueeess in the AFIT GEEM program by AFIT GEEM 
administrators. 

The following seetions present researeh on the predietive validity, strengths, and 
weaknesses of the AFIT GEEM admissions eriteria. 

Undergraduate Grade point Average (UGPA). Betts and Morel! (1998) define 
the UGPA as a measure of a student’s undergraduate aeademie sueeess. Mathematieally 
defined, the UGPA is a ratio of total quality points, the product of earned grade points 
and credit hours, to the total credit hours taken by the student. Virtually every 
undergraduate university and college in the United States calculates an UGPA for its 
students, with very little variance in methodology. Grade points, most commonly, range 
from 0.0 to 4.0, where an ‘F’ reflects a 0.0 and an ‘A’ is a 4.0. The undergraduate 
institution uses the UGPA for a variety of purposes, including the award of academic 
distinctions and the determination of graduation eligibility. Most undergraduate 
institutions require at least a 2.0 or ‘C’ UGPA for graduation. Graduate institutions, 
however, use the UGPA to make inferences about an applicant’s ability to perform in 
graduate school, where graduate performance is measured in a variety of ways (— i.e., 
first-year GGPA, cumulative GGPA, and graduation). 
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Researchers disagree on the ability of the UGPA to validly fill this role. UGPA 
validity studies examine specific academic departments or across academic departments 
and universities. In either case, no consensus on the predictive validity of the UGPA 
exists. For example, in a study of MBA graduates from a southeastern US business 
school, Ahmadi, Raiszadeh, and Flelms (1997) found a strong positive correlation (r = 

.52) between UGPA and GGPA. In a regression-based study of graduate, criminal justice 
students at a southern US university, McKee, Mallory and Campbell (2001) found that 
the UGPA, alone, accounted for twenty-four percent of the variance in GGPA (r = .49). 
McKee, Mallory and Campbell (2001) also reported that GRE (verbal), GRE 
(quantitative), and GRE (analytical) scores predicted an additional sixteen percent of the 
variance in GGPA. Performing a stepwise multiple regression analysis on three groups 
of graduate students entering UCLA graduate schools in 1981, 1983, and 1985, Abedi 
(1991) found junior/senior UGPA to have “virtually no relationship with any of the 
measures of graduate academic success” (p. 160), defined as a composite measure 
consisting of “GGPA, the total number of credit hours attempted, and the type and 
number of degrees earned” (p. 154). Shelton (2001) explains that the UGPA accounts for 
“less than seven percent of the variance in first-year law school grades” (p. 1). 

While research disagrees on the predictive validity of the UGPA, many 
researchers have proposed explanations for the inconsistencies. Goldberg and Alliger 
(1992:1026) and Malone, Nelson and Nelson (2001:4) explain that undergraduate grades 
exist within a tight range of values. As a result, differences in UGPA may not accurately 
reflect differences in the magnitude of student accomplishment. Malone, Nelson and 
Nelson (2001) explain that “it is not always clear what grades mean” (p. 4). The grade 
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could be a measure of the student’s course performance, performance compared to the 
student’s peers, or a measure of the student’s performance improvement since the 
beginning of the class (Malone, Nelson, and Nelson, 2001:4). Goldman and Slaughter 
(1976) and Morrison and Morrison (1995) blame a lack of standard practices used in 
assigning grades for dramatic grade variance within classes, across academic disciplines, 
and across different institutions. Malone, Nelson and Nelson (2001) further explain, 
“grade assignments are also sometimes influenced by student characteristics that have no 
relationship to academic performance - i.e., participation in class, argumentative 
behavior” (p. 5) (Singer 1964). Grade inflation may also prove to be a factor that 
compromises the predictive validity of the UGPA. Studying 134 colleges, Juola (1977) 
reports a mean average increase of M = 0.40 grade points from 1965 to 1973. Juola 
(1980) later expanded the study to include 180 undergraduate institutions and found a 
mean average increase of M = 0.43 grade points from 1960 to 1979. Levine (1994) 
surveyed 4900 college graduates from the classes of 1960 through 1993 and found a 
sixty-six percent increase in the number of A’s given. The inflation of grades further 
restricts the already “narrow range” (Malone, Nelson, and Nelson; 2001:4) of 
undergraduate grades, further masking differences in student academic performance that 
help distinguish between applicants admitted and applicant rejected (Malone, Nelson, and 
Nelson; 2001:4). 

In addition to independent studies, the Educational Testing Service (ETS) has 
published a table of average estimated correlation between UGPA and first-year graduate 
school grades in the 2000-2001 Guide to the Use of Scores GRE Manual. The ETS 
results tend to bring stability to the findings of the independent research. Across 1,038 
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academic departments and 12,013 examinees, the ETS reports an r = .37 correlation 
between UGPA and FYGGPA. The ETS also reports correlation results for specific 
academic departments. For engineering programs, looking across eighty-seven 
departments and 1,066 examinees, the ETS published an r = .38 correlation between 
UGPA and FYGGPA. The ETS reports similar findings among similarly sized samples 
in the Natural Sciences (r = .37), Social Sciences (r = .38), Flumanities and Arts (r = .37), 
Education (r = .35), and Business (r = .39). 

A significant body of research also exists on the predictive validity of the UGPA 
in combination with other criteria. For example, Malone, Nelson, and Nelson (2001) 
found the UGPA to be a valid predictor of graduate school success, defined 
dichotomously as graduated or did not graduate, when multiplied by the score from the 
verbal section of the GRE. Malone, Nelson, and N elson (2001) analyzed this combined 
variable in consideration of “Michael’s (1983) findings that combinations of predictor 
variables yielded higher validity than single predictors” (p. 13). In the 2000-2001 Guide 
to the Use of Scores , the ETS published an average estimated correlation of the product 
of UGPA, GRE-V, GRE-Q, and GRE-A and FYGGPA of r = .46, an improvement of Ar 
= .09 over the correlation between UGPA and FYGGPA. For engineering students, the 
average estimated correlation of the product of UGPA, GRE (verbal), GRE (quantitative), 
and GRE (analytical) and FYGGPA was r = .44, an ^ = .06 improvement over the 
correlation between UGPA and FYGGPA. 

Beyond combining the UGPA with other predictors, Schneider and Briel (1990) 
indicate that UGPA correlated with first-year GGPA in a similar manner as the GRE 
Subject Test score and first-year GGPA. The 2000-2001 ETS study generally supports 
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this report. For engineering students, looking across twenty-one departments and 185 
examinees, the correlation between UGPA and first-year GGPA is r = .39. Across the 
same sample, the ETS published an r = .41 correlation between first-year GGPA and the 
Engineering Subject Test. The similar correlation between UGPA and first-year GGPA 
and GRE Subject Test and first-year GGPA holds for most of the other academic 
departments. Chemistry and Economics, however, showed large differences — Ar = .15 
and Ar = .05, respectively — between UGPA and first-year GGPA correlation and GRE 
Subject Test and first-year GGPA correlation. 

From the preceding presentation of research on the predictive validity of the 
UGPA, it is clear that there is a great deal of variability among the focus and findings of 
individual studies. Such inconsistency only justifies the performance of more validation 
studies to determine the predictive validity of the UGPA within the specific context of the 
universities or academic departments that use the measure to evaluate applicants for 
admission. 

Graduate Record Examination (GRE) General Test. Kuncel, Flezlett, and Ones 
(2001) state that the Graduate Record Examination was “developed in the 1940’s” 
(p.l62). Since 1966, the Graduate Record Examination (GRE) Board has been charged 
with improving the measure to “equalize higher education opportunities for all students, 
improve the practices, procedures, and quality of graduate education, and promote 
maximum utilization of human talents and financial resources” (ETS Guide to the Use of 
Scores. 2000-2001:5). The efforts of the GRE Board have most notably translated into 
the addition of the analytical section, beyond the original quantitative and verbal 
batteries, the implementation of the computer-adaptive test (CAT), and, most recently. 
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the consideration of the addition of a writing sample. Yet, despite the modifications 
made to the GRE, “the results of this half century of research” on the GRE’s validity 
“have been inconsistent and controversial” (Kuncel, Hezlett, & Ones, 2001:163). This is 
proof that the GRE is still not a perfect measure and will continue to evolve in order to 
“serve the best interests of the entire graduate community” (ETS Guide to the Use of 
Scores, 2000-2001:5). 

In the 2000-2001 Guide to the Use of Scores, the ETS claims that the Graduate 
Record Examination (GRE) General Test measures the graduate school applicant’s 
accumulated verbal, quantitative, and analytical skills on a common scale. Scores for 
each of the three components range from 200 to 800 points. The verbal component 
measures the applicant’s ability to analyze written passages as well as the relationship 
among sentences in the passage and the relationships among the words within the 
sentences. The quantitative section tests the ability to use basic mathematical concepts to 
solve quantitative problems. The analytical test examines the ability to both infer and 
deduce information from a given set of relationships — i.e., analyzing arguments. 

While verbal, quantitative, and analytical skills are measured using separate 
sections, there is a moderate-to-high overlap in the skills tested in each section. In the 
2000-2001 Guide to the Use of Scores, the ETS reports average estimated correlation of r 
= .45 between the verbal and quantitative sections, r = .66 between the quantitative and 
analytical components, and r = .60 between the verbal and analytical components. Rock, 
Werts, and Grandy (1982) report correlations of r = .64 between verbal and quantitative 
sections, r = .77 between quantitative and analytical sections, and r = .77 between the 
verbal and analytical components. 
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According to Norcross, Hyanch and Terranova (1996), eighty-one pereent of 
psyehology graduate programs require applieants to sit for the GRE General Test. At test 
centers across the world, the Educational Testing Service (ETS) administers the 
computer-based adaptive GRE General Test throughout the year. The computer-adaptive 
test (CAT) model selects questions according to difficulty and examinee responses on 
past questions within the parameters of required question-type variety and content 
coverage. While few test centers still administer the original paper-based GRE General 
Test, the ETS reports that scores on the computer-adaptive tests are comparable to those 
on paper-based tests. 

According to the ETS’s 2000-2001 Guide to the Use of Scores, graduate 
institutions may appropriately use General Test scores “to admit graduate students, select 
graduate fellowship students for awards, select teaching or research assistants, or to 
perform graduate study counseling” (p. 8). The ETS does not consider the GRE to be 
suitable for other uses. Regardless of the suitability of the application, the ETS 
recommends that graduate institutions never use the General Test in isolation. General 
Test scores do not exactly and completely measure all the factors that influence the 
prediction of graduate school success. The ETS reports that the GRE General Test, 
alone, possesses only slight-to-moderate ability to predict first-year GGPA and tend to 
vary according to academic discipline. For example, while the first-year GGPA 
correlated positively with the verbal section (r = .30), quantitative section (r = .29), the 
analytical section (r = .28) and the GRE General Test Total score (r = .34) across 1,038 
departments (12,013 examinees), the ETS reported that the predictive validity of the 
components varied according to academic department. For example, a study of eighty- 
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six education departments (901 examinees) produced an average estimated eorrelation of 
r = .31 (verbal), r = .30 (quantitative), r = .29 (analytical), and r = .36 (General Test total 
score). Yet, a study of eighty-seven (1,066 examinees) engineering departments 
produced an average estimated correlation of r = .27 (verbal), r = .22 (quantitative), r = 
.24 (analytical), and r = .30 (General Test total seore) (ETS Guide to the Use of Seores. 
2000-2001:24). Studies by Braun and Jones (1985), Thomell and MeCoy (1985), and 
Morrison and Morrison (1995) report evidence of variability in the predictive validity of 
the GRE across academic disciplines. Thomell and MeCoy (1985) report that across four 
academic disciplines - education, humanities, fine arts, and math/seience - the validity 
coefficients of GRE (quantitative) and GGPA varies from r = .22 for fine arts to r = .37 
for math/science. The validity coefficients for GRE (total seore) and GGPA vary from r 
= .36 for fine arts to r = .48 for math/seience. The validity coefficients of GRE (verbal) 
and GGPA vary from r = .42 for fine arts to r = .49 for education. Morrison and 
Morrison (1995) report weighted averages of observed effect sizes for GRE (quantitative) 
when GGPA was the measure of graduate school performance ranging from d = -.07 to d 
= 1.7. Morrison and Morrison (1995) also reported that the weighted average observed 
effect sizes for GRE (verbal) and GGPA varied from d = -.26 to d = 2.2. The weighted 
average observed effect sizes for GRE (total) and GGPA varied from d = -.1 to d = 1.3. 
Thus, the ETS recommends that graduate institutions use the GRE General Test as a 
“supplement” (p. 5) to other predictors of graduate school success, such as the UGPA and 
letters of recommendation. If graduate institutions plan on using the GRE for a suitable 
purpose, the ETS encourages individual universities and academic departments to 
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conduct in-house validity studies of the GRE General Test (ETS Guide to the Use of 
Scores. 2000-2001:7). 

The results of those in-house studies tend to vary with academic discipline and 
sometimes produce mixed results within disciplines. For example, while Wilson (1979) 
found median validity coefficients for first-year grades in psychology of r = . 18 (verbal) 
and r = .19 (quantitative), Flouse, Johnson, and Tolone (1987) found a correlation 
coefficient of r = .15 for GGPA in psychology. The 2000-2001 ETS Guide to the Use of 
Scores reports a r = .29 (verbal), r = .29 (quantitative), and r = .28 (analytical) correlation 
with first-year GPA in psychology. Goldberg and Alliger (1992) performed a meta¬ 
analysis of ten studies, focusing on graduate psychology students, and found “somewhat 
lower” (p. 1025) correlation values that those reported by the ETS: r = .15 (verbal) and r 
= .14 (quantitative). Burton & Turner (1983) report validity coefficients for first-year 
grades across all social sciences of r = .26 (verbal) and r = .22 (quantitative). Flowever, 
Milner, McNeil, and King (1984) reported that the GRE was not a valid predictor of 
GGPA in a study of social work students. Dunlap (1979) corroborates the Milner, 
McNeil, and King (1984) finding, stating that the GRE was a “weak predictor” of 
graduate school performance. Flowever, the 2000-2001 Guide to the Use of Scores. 
published by the ETS, reports average estimated correlations of the GRE General Test 
scores with graduate first-year GPA of r = .33 (verbal), r = .32 (quantitative), and r = .30 
(analytical) for the social sciences. Morrison and Morrison (1995) performed a meta¬ 
analysis of twenty-two studies. They reported that the GRE demonstrated minimal 
predictive validity of GGPA, with a r = .22 quantitative validity coefficient and a r = .28 
verbal validity coefficient. 
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One study, however, eondueted by Kuneel, Hezlett, and Ones (2001), stands 
against the elaim that the predietive validity of a eomprehensive meta-analysis, 
examining 1,521 studies with 82,659 graduate students. They found all three eomponents 
of the GRE General Test to be “generalizably valid predietors of GGPA, E*-year GGPA, 
faeulty ratings, eomprehensive examination seores, eitation eounts, and, to a lesser extent, 
degree attainment” (Kuneel, Hezlett, & Ones, 2001:174). On their findings, Kuneel, 
Hezlett and Ones (2001) explain that “prior eritieisms of the GRE’s validity as 
situationally speeifie and useless are in error” (p. 176). 

Control Variables. There are many faetors that infiuenee the student’s UGPA, 
GRE seore, and UMGPA. For example, age, gender, and the differenee (in years) 
between undergraduate and graduate education (TDELTA) have been shown to influence 
the predictive validity of the UGPA and the GRE General Test. Matthews and Martin 
(1992) reported that age interacts with predictor variables to cause underpredictions for 
both older, low-to-moderate credentialed students - i.e, those over thirty years of age - 
and younger, high-credentialed students. House (1989) reports that GRE scores 
overpredicted the graduate school performance of younger students - those students 
under twenty-four years of age — and underpredicted the graduate school performance of 
older students - those students over the age of twenty-five. On gender, Payne, Wells and 
Clarke (1971) found that females tend to be “more predictable than males” (p. 498). 
Computing multiple correlations from the combination of National Teacher Examination 
score (NTE-C), GRE-T (total of quantitative and verbal scores), and UGPA; Payne, 

Wells and Clarke (1971) achieved multiple R’s of .34 (N = 61), .45 (N = 66), and .32 (N 
= 127) for males, females, and the total sample, respectively. House (1994) found the 
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correlation between the total GRE seore and GGPA was signifieantly stronger for 
women. House (1994) also found the eorrelation between the verbal GRE seore and 
GGPA to be significantly stronger for women. However, the research on the influence of 
age, gender, and TDELTA on the predictive validity of UGPA and the GRE General Test 
does not present consistent findings across studies. For example, Malone, Nelson and 
Nelson (2001) concluded that both age and TDELTA had no predictive power after being 
eliminated from the regression model used in their study (p.l 1). 

Whether or not age, gender, and TDELTA have true predictive power is not a 
problem that must be solved or theorized before research and analysis can begin. When 
seeking to validate admissions criteria, like the UGPA and the GRE General Test, for 
example, the researcher is most directly concerned with the predictive validity of these 
measures. If age, gender, and TDELTA exert any influence on the prediction of graduate 
school success using these or any other measures, then controlling for them provides the 
researcher with a way to measure the contributions of UGPA and the GRE General Test 
beyond the contributions of age, gender, and TDELTA. Controlling for these variables, 
despite whatever mixed findings may characterize the research, cannot harm the research 
effort. If age, gender, and TDELTA turn out to have, in fact, no predictive power of 
influence on other predictors, then controlling for them in a regression model will not 
hinder the predictive power of the other variables. 
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Graduate School Success 


Malone, Nelson and Nelson (2001) explain that the definition of graduate sehool 
sueeess is a “eoneept without preeise definition,” (p. 7). Researehers have most 
eommonly defined graduate sehool sueeess as either FYGGPA, GGPA, or 
diehotomously, as graduated versus did not graduate. Goldberg and Alliger (1992) list 
some less eonventional definitions; ineluding “graduate fellow sueeess, eomprehensive 
examination seores, faeulty ratings, and grades in speeifie eourses” (p. 1021). 

However, both the eommon and uneommon definitions of graduate sehool 
sueeess have demonstrated mixed relationships with seleetion eriteria. Goldberg and 
Alliger (1992) found that the GRE, as a predietor of graduate sehool sueeess, explained 
more varianee in eomprehensive examination seores, the measure of graduate sehool 
sueeess, than any other predietor studies in their meta-analysis of ten studies. However, 
they warn that such a high validity may have been the product of a “circular 
relationship,” where “a test score is being used to predict another test score” (p. 1024). 
The ETS reports that the GRE General Test, alone, possesses only slight-to-moderate 
ability to predict FYGGPA. Yet, the ETS found better results when the GRE General 
Test scores, the UGPA, and the GRE Subject Test score were all used as predictors of 
FYGGPA (ETS Guide to the Use of Scores . 2000-2001:24). Abedi (1991) found that 
UGPA was “virtually not related” (p. 160) to the composite measure of graduate school 
success, consistent of the number of credit hours taken, GGPA, and the type and number 
of degrees earned. Goldberg and Alliger (1992) found practically no predictive 
relationship between the GRE verbal and analytical sections and specific course grades. 
The GRE quantitative section proved slightly more predictive of course grades, but 
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Goldberg and Alliger (1992) believe a moderator variable to be effecting the relationship. 
Milner, McNeil and King (1984) reported that the GRE was not a valid predictor of 
GGPA in a study of social work students. In a study of MBA graduates from a 
southeastern US business school, Ahmadi, Raiszadeh and Helms (1997) found a strong 
positive correlation (r = .52) between UGPA and GGPA. 

Goldberg and Alliger (1992) explain that researchers must first improve 
definitions of graduate school success before they can expect to make significant findings 
on the validity of the GRE. In fact, Goldberg and Alliger (1992) claim that the 
definitions must improve before making significant findings on any predictor of graduate 
school success. To improve graduate school success definitions for graduate psychology 
programs, Goldberg and Alliger (1992) advise researchers to focus on “what we are 
ultimately trying to predict, graduation and scientific productivity” (p.l026). For 
different graduate programs, however, the definition of success may emphasize different 
criteria depending on the goals of the program. Engineering and psychology programs 
have very different curriculums and, quite arguably, very different goals. If engineering 
does, in fact, so differ from psychology, then student success in each discipline should 
not be measured by the same standard. 

The Present Study 

This study examines the AFIT admission criteria as predictors of graduate school 
success and controls for age, gender, and time (in years) between undergraduate and 
graduate education. Since the completion of at least one undergraduate differential 
equations course is required for admission to the AFIT GEEM program, most every 
student (N = 146) in the GEEM database had taken such a course. Therefore, whether the 
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student had taken a differential equations eourse (eoded ‘ 1 ’ if the student had taken a 
eourse and ‘0’ if the student had not taken a eourse) was not eontrolled for in this study. 

The AFIT Graduate Catalog for the Aeademie Year 2001-2002 defines graduate 
sehool sueeess via two requirements for graduation: at least a 3.00 (on a 4.00 seale) 
GGPA and the eompletion of a thesis. However, while simply meeting these 
requirements may be suffieient for a student to be sueeessful at AFIT, this study requires 
that graduate sehool sueeess be defined more intrieately; for nearly 100% of AFIT 
students graduate within the allotted eighteen months. Thus, this study distinguishes 
between relative levels of graduate student sueeess, as measured by GGPA, Coursework 
GGPA and thesis grade. Students with higher GGPAs and thesis grades are deemed 
more sueeessful than those with relatively lower GGPAs or and thesis grades. 

Considering that the ETS uses the FYGGPA to validate the GRE examination, 
this study will eonsider FYGGPA as an alternative definition of graduate sehool sueeess. 
Again, students with higher FYGGPAs are deemed more sueeessful than those with 
relatively lower FYGGPAs. 
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Methods 


In studies that examine the predictive validity of admission requirements - i.e., 
UGPA, GRE General Test scores, etc. - bivariate correlation and multivariable regression 
analysis are the most common methodologies. This study employs both of those 
methodologies to investigate the predictive validity of the AFIT admission requirements. 

This chapter opens with a summary description of the population used in this 
study and a discussion of the significant differences in dependent variable mean values 
according to gender, race, age, rank and undergraduate institution. The procedure used to 
collect the data and conduct a preliminary review for entry errors follows, focusing on the 
admissions requirements and measurements of graduate school success. This chapter 
closes with a description of the bivariate correlation and multivariable regression 
procedure. 

Population and Procedures 

This study collected data on the 173 students that attended the AFIT GEEM 
program from 1995 to 2002, a period over which the structure, content and duration of 
the GEEM program was stable. Ninety-one percent of those students (N = 158) were US 
Air Force Officers. The remaining sixteen students were US Marine Corps Officers (N = 
7), DoD Civilian employees (N = 8). For US military officers, student service rank 
ranged from second lieutenant and major. Military ranks have corresponding numbers - 
i.e., a second lieutenant is an 0-1 (officer grade 1). The mean average of the numeric 
military ranks is M = 2.36, indicating that military officers in this population tend to be 
between the rank of first lieutenant and captain. The civilian pay-grades ranged from 
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GS-09 (Government Service 09) to GS-13 with a fairly even spread of employees across 
grades. The civilian pay-grades correspond to military officer ranks as follows: 

GS-09 = 0-1, GS-11 = 0-2, GS-12 = 0-3, GS-13 = 0-4 

The mean student age at entry to the AFIT GEEM program was M = 27.62 years 
(SD = 3.38). The youngest student was twenty-two years old at entry, and the oldest 
student was forty-eight years old at entry. GEEM students took a mean average M = 4.27 
years (SD = 2.96) between undergraduate and graduate education. Eighty-seven percent 
of the population (N = 150) was male. Eighty-nine percent were Caucasian (N = 154). 
Eight were African-American, and eleven were classified as “other,” which includes both 
Hispanics and Asians. 

Data collected on the undergraduate institutions students attended revealed that 
twenty-seven percent of the students received their bachelor’s degree from the US Air 
Force Academy (47 of the 173). The Texas A&M University posted the second greatest 
representation with five students. The three most common Bachelor of Science degrees 
among students were Civil Engineering (79 students). Mechanical Engineering (27 
students), and Electrical Engineering (24 students). The remaining forty-three students 
possessed Bachelor of Science degrees in a variety of science and engineering fields. 
Tables of the undergraduate institutions attended and the degrees earned by AFIT GEEM 
students are contained in Appendices A and B, respectively. 

All data were collected from student education records maintained by the AFIT 
registrar. Access to these files can be gained only by written permission of the director of 
the AFIT registrar. A copy of the letter requesting access to the AFIT academic records 
has been placed in Appendix C. 
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The academic files usually contained undergraduate transcript(s), a military 
service summary sheet, and the AFIT transcript. If one of these documents was absent 
from a student’s file and a particular data point could not be determined from any other 
document in the student’s record, then the data point was left blank. Thus, some data 
elements may have fewer data points than other data elements. 

Measures 

Undergraduate Grade-Point Average (UGPA). Undergraduate transcripts 
provided the student’s cumulative academic performance at that particular institution. 
Mathematically defined, the UGPA is a ratio of total quality points, the product of earned 
grade points and credit hours, to the total credit hours taken by the student. Grade points 
were assigned consistent with AFlT’s standard, where (AFIT GSEM Graduate Catalog, 
Academic Year 2001-2002, p. 22): 

A = 4.0, A- = 3.7, B+ = 3.3, B = 3.0, B- = 2.7, C+ = 2.3, C = 2.0, D = 1.0. F = 0.0. 

There were only a few deviations from this practice. For example, Worcester 
Polytechnical Institute (WPl) used a single letter grade to cover the typical B/C range. 
While an ‘A’ - Excellent - was labeled as ‘DIST/AD,’ the grade ‘AC’ was given to both 
a ‘B’ - Good - and a ‘C’ - Acceptable. Although the AFIT academic evaluator produced 
a UGPA for this student, the author did not find any procedure for converting the ‘AC’ 
grade into grade points. The UGPA could not be verified and was, therefore, not factored 
into the database for the WPl (N = 1) student. 

In other cases, some students attended more than one undergraduate school before 
receive their bachelor’s degree. In such cases, a composite UGPA, the ratio of the total 
quality points earned at all undergraduate institutions to the total credit hours earned 
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across those undergraduate institutions, was ealeulated and subsequently used in the 
analysis. 

Undergraduate Math Grade-Point Average (UMGPA). The UMGPA was 
calculated in the same manner as the composite UGPA, using the institute’s published 
grading standard. The UMGPA was derived using any and all math-related classes 
published on a student’s undergraduate transcript(s). Courses in Analytical Geometry, 
Analytical Geometry with Calculus, College Algebra, Trigonometry, Pre-Calculus, 
Integral Calculus, Differential Calculus, Ordinary Differential Equations, Applied 
Differential Equations, Probability and Statistics, as well as any course carrying the same 
department code as the aforementioned courses were factored into the UMGPA. The 
most common department codes were ‘MTH.’ And ‘MA.’ 

Graduate Record Examination (GRE) General Test Scores. GRE General Test 
scores were collected from the official ETS report submitted to AEIT for each of the 
three components: verbal (GRE-V), quantitative (GRE-Q), and analytical (GRE-A). 
However, AFIT requires applicants to submit only the quantitative and verbal scores. If 
applicants reported their GRE scores to the AFIT admission office other than by the 
official ETS report, then the analytical scores were, most likely, not included. Of the 173 
students in this study, eleven do not have analytical scores. 

If more than one GRE score appeared in the student’s record, the most recent test 
was used. In the event that no date was attached to the GRE scores, then the highest 
summed GRE score (verbal and quantitative components only) was used. 

Graduate Grade-Point Average (GGPA). The GGPA was taken directly from the 
AFIT transeript. The GGPA was the cumulative grade point average after six-quarters of 
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study at AFIT. The GGPA also includes the grade for the twelve hours of thesis study. 
The GGPA was used both with and without the influence of the thesis grade. A 
histogram of GGPA and Coursework GGPA values appears in Appendix D.l, Figures 1 
and 2, respectively. 

Thesis grade. The thesis grade was reported on the AFIT transcript in the sixth 
and final quarter of study. All AFIT theses represent twelve quarter-hours of credit. A 
histogram of thesis grade values appears in Appendix D.2, Figure 3. 

First-Year Graduate Grade-Point Average (FYGGPA). The student’s quarterly 
GGPAs were collected from the AFIT transcript. Flowever, the hours and quality points 
that produced the quarterly GGPAs were not collected. Thus, when the FYGGPA was 
added to the study as a dependent variable, the data necessary to calculate a cumulative 
FYGGPA had not been collected. Therefore, the FYGGPA was calculated as the mean 
GGPA of the first four quarters of study at the AFIT. For future studies, however, it is 
recommended that the FYGGPA be the cumulative GGPA after the first four quarters of 
graduate study. A histogram of FYGGPA values appears in Appendix D.2, Figure 4. 
Review of Data 

While data were being collected, it was checked for entry errors against the 
original document. Once the database was complete, a cursory, visual inspection of the 
data was performed. The mean and variance of numeric data elements were used to 
perform a reality check on the data, looking for incorrectly entered data points that were 
influencing the two population parameters. Those variables that could not be averaged - 
i.e., race - were counted to both ensure that the number of data points collected did not 
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exceed the total number of students in the population and that a sufficient number of data 
points were collected to satisfy central limit theorem assumptions. 

Any data-entry mistakes discovered by these precautionary measures were 
corrected by consulting the academic record of that particular student. For example, the 
initial variance of TDELTA was about 99 years. TDELTA was calculated by taking the 
difference between the year the student started AFIT and the year the student graduated 
from the undergraduate program. An inspection of the data on TDELTA revealed that 
two data cells incorrectly contained the value ‘1994.’ This error was unduly influencing 
the variance, and also, although less obvious, the mean of TDELTA. The student’s 
academic files were reviewed for both the year the student entered AFIT and the year the 
student graduated from their undergraduate program. The correct years were entered into 
the database and the correct mean and variance were calculated. 

Regression Modeling 

SPSS 10.0 software was used to perform multivariable, linear, hierarchical 
regression analyses. For each dependent variable, GGPA, Coursework GGPA, thesis 
grade, and FYGGPA; three separate hierarchical analyses, with either two or three steps, 
were performed. The first step of each of the three analyses regressed only the control 
variables, age, gender, and time (T) between undergraduate and graduate education 
(TDELTA), against the dependent variable. Beyond the controls, the second step of each 
analysis measured a different predictor variable against the dependent variable. One 
analysis entered UGPA in the second step, measuring its ability to predict graduate 
school success beyond the control variables. The next analysis placed both GRE (verbal) 
and GRE (quantitative) in the second step. The final analysis placed UMGPA in the 
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second step and all of the remaining predictors - UGPA, GRE (verbal), and GRE 
(quantitative) - in the third step. 

SPSS 10.0 generated descriptive statistics for the data elements used in the 
analysis. In addition, a model summary, which contained the R and the AR values for 
the regression, and coefficients table, which contained the standardized |3 coefficients, 
were produced. The software also produced a normal probability plot, a scatter plot of 
the standardized predicted residuals versus the standardized residuals, and VIF 
multicollinearity factors to check the assumptions of the General Linear Model (GEM). 
Finally, a Pearson correlation table was generated, using a p = 0.05 to evaluate 
significance of relationships. 

Using the normal probability plot, scatter plot, and VIF multicollinearity factors, a 
quick check of the GLM assumptions was performed. Beyond the assumptions, 
significant Pearson correlations (p < .05) were used to identify bivariate predictive 
capability among the independent, dependent, and control variables. Significant AR (p < 
.05) and significant standardized |3 coefficients (p < .05) were used to identify statistically 
significant predictors of graduate school performance. 

SPSS 10.0 software was also used to perform one-way ANOVA to explore 
significant differences in dependent variables according to age, gender, race, rank, and 
undergraduate institution. 
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Results 


Regression Assumptions 

Neter, Kutner, Nachtsheim, and Wasserman outline the assumptions of the 
General Linear Model (GLM). The residuals are assumed to be independent, normally 
distributed, and have a eonstant variance. 

The GGPA, Coursework GGPA and FYGGPA regression models satisfied all of 
the GLM’s assumptions. Scatter plots showed the standardized residuals for the GGPA, 
coursework GGPA and FYGGPA models to be homoscedastic. Normal probability plots 
demonstrated that the standardized residuals are normally distributed. For both models, 
VIF factors were examined to determine if multicollinearity exists between the variables. 
Neter, et ah, advise that VIF factors greater than ten indicates significant multicollinearity 
among variables. Flowever, for the GGPA, coursework GGPA and FYGGPA models, 
VIF factors did not exceed four. The highest VIF factors occurred between age and 
TDELTA, which seems reasonable considering the significant bivariate correlation 
between the two variables (r = .77, p < .05). 

The thesis grade model, on the other hand, did not satisfy the homoscedasticity 
and normal error distribution assumptions, as demonstrated by a scatter plot and normal 
probability plot, respectively. The thesis grade model VIF factors, like those for the 
GGPA, coursework GGPA and FYGGPA models, did not exceed four. The highest VIF 
factors occurred between age and TDELTA. 


30 



Descriptive Statistics 

In Table 4.1, mean, standard deviations, and eorrelations for the independent, 
dependent and eontrol variables are presented. 

Applieants eligible for study in the GEEM program must have an earned 
baeealaureate degree from an aeeredited eollege or university in an appropriate diseipline 
- most eommonly eivil, meehanieal, or eleetrieal engineering - an overall undergraduate 
grade point average (UGPA) of at least 3.00, and Graduate Reeord Examination (GRE) 
General Test seore of at least 500 on the verbal seetion and 600 on the quantitative 
seetion. In addition, the AFIT GEEM program uses the undergraduate math grade point 
average (UMGPA) as an indieator of the level of mathematieal study and profieieney. 

The 3.00-standard applies to the UMGPA eriterion, as well. Applieants must also have 
taken an undergraduate eourse in ordinary differential equations (ODE). 

Considering the standard deviations about the mean values of the UGPA (M = 
3.02, SD = 0.30), the UMGPA (M = 2.91, SD = 0.50), and the GRE (verbal) (M = 

525.03, SD = 76.75); it was elear that AFIT admits students with grades or seores below 
that of the stated minimums. For example, a ninety-five pereent eonfidenee interval 
around the mean UGPA ranged from 2.53 to 3.52. Similar intervals were ealeulated 
around the means of UMGPA (from 2.09 to 3.74), and GRE (verbal) (from 398.39 to 
651.67). The only variable that did not drop far below the AFIT-stated minimum was the 
GRE (quantitative) seore (M = 689.26, ^ = 55.60), whieh seemed reasonable 
eonsidering that the AFIT GEEM program was an engineering-based program. A ninety- 
five pereent eonfidenee interval around the mean GRE (quantitative) ranged from 597.52 
to 781. 
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The GGPA, coursework GGPA and FYGGPA had mean values of M = 3.69 ISD 
= 0.16), M = 3.69 (SD = 0.17) and M = 3.67 (SD = .17), respeetively. A ninety-five 
pereent eonfidenee interval about the mean of GGPA ranged from 3.43 to 3.95. A 
ninety-five pereent eonfidenee interval about the mean of eoursework GGPA ranged 
from 3.41 to 3.97. A ninety-five pereent eonfidenee interval about the mean of FYGGPA 
ranged from 3.39 to 3.95. There were signifieant eorrelations between GGPA and 
FYGGPA (r = .93, p < .05) and eoursework GGPA and FYGGPA (r = .97 p < .05). Both 
the proximity of the means and the high eorrelation between these three dependent 
variables was not eoineidental. The FYGGPA was merely a eomponent of both the 
GGPA and the eoursework GGPA. The FYGGPA was the average of the first four 
quarters of grades that eontribute to both the GGPA and eoursework GGPA. 

There were signifieant eorrelations between GGPA and Age (r = .14, p < .05), 
TDELTA (r = .18, p < .05), UGPA (r = .18, p < .05), GRE (verbal) (r = .25, p < .05), and 
GRE (quantitative) (r = .15, p < .05). There were signifieant eorrelations between 
coursework GGPA and Age (r = .24, p < .05), TDELTA (r = .23, p < .05), UGPA (r = 

.14, p < .05), GRE (verbal) (r = .30, p < .05), and GRE (quantitative) (r = .24, p < .05). 
There were no significant correlations between any of the independent or control 
variables and thesis grade. Age (r = .20, p < .05), TDELTA (r = .24, p < .05), UGPA (r = 
.17, p < .05), GRE (verbal) (r = .29, p < .05), and GRE (quantitative) (r = .20, p < .05) 
and were all significantly correlated with FYGGPA. 

There was a significant correlation between the GRE General Test verbal and 
quantitative sections (r = .38, p < .05). A significant correlation existed between the 
analytical section and the verbal section (r = .38, p < .05). A significant correlation 
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existed between the quantitative and the analytical section (r = 0.45, £ < .05). These 
results were lower than those reported in the 2000-2001 Guide to the Use of Scores. 
where the ETS found average estimated correlation of r = .45 between the verbal and 
quantitative sections, r = .66 between the quantitative and analytical components, and r = 
.60 between the verbal and analytical components. These results were comparable to the 
correlations reported by Sternberg and Williams (1997). Sternberg and Williams (1997) 
found correlations of r = .26 (p < .01) between verbal and quantitative sections, r = .47 (p 
< .001) between quantitative and analytical sections, and r = .49 (p < .001) between the 
verbal and analytical components. This study’s results were lower than the correlations 
reported by Rock, Werts, and Grandy (1982). Rock, Werts, and Grandy (1982) found 
correlations of r = .64 between the verbal and quantitative sections, r = .77 between the 
quantitative and analytical sections, and r = .77 between the verbal and analytical 
components. 
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Descriptive Statistics and the Lower Diagonal of Intercorrelations Among Study Variables 



TT 






One-Way ANOVA 

In Table 4.2, significant differences in the mean values of each of the dependent 
variables was investigated according to gender, race, rank, and undergraduate institution 
using one-way ANOVA. 

When one-way ANOVA was performed to investigate the significance of 
differences in mean dependent values according to gender, no significant differences 
were found for any of the four dependent variables. 

When one-way ANOVA was performed to investigate the significance of 
differences in mean dependent values according to race, no significant differences were 
found for any of the four dependent variables. 

When one-way ANOVA was performed to investigate the significance of 
differences in the mean dependent values according to rank, significant differences were 
found between the dependent variable means for first lieutenants and captains for GGPA 
(AM = -0.07, E < .05), coursework GGPA (AM = -0.10, e < -05), and FYGGPA (AM = - 
0.09, E < -05) dependent variables except thesis grade. 

When one-way ANOVA was performed to investigate the significance of 
differences in the mean dependent values according to whether or not the student 
attended a US Service Academy (USAFA: N = 47, USNA: N = 2), it was found that 
Service Academy graduate significantly had lower GGPA (AM = -0.06, p < -05), 
coursework GGPA (AM = -0.06, p < -05), and FYGGPA (AM = -0.07, p < .05) than 
student who attended other undergraduate institutions. 
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Table 4.2 


One-Way ANOVA Results for Demographic Variables (Gender. Race. Rank, and 
Undergraduate Institution) Against Measures of Success 






Denendent Variables 





GGPA 

Thesis Grade 

Coursework GGPA 

FYGGPA 


M 

SD 

M 

SD 

M 


M 


Gender 

Male 

3.69 

.16 

3.71 

.31 

3.69 

.15 

3.66 

.17 

Female 

3.70 

.14 

3.75 

.30 

3.69 

.17 

3.70 

.15 

ANOVA Results 

F(l,151) 

= 0.17 

F(l,149) 

= 0.29 

F(U150) 

= 0.00 

F(l,147) 

= .079 

Race 

White 

3.69 

.16 

3.72 

.30 

3.69 

.17 

3.67 

.17 

Black 

3.64 

.15 

3.59 

.46 

3.65 

.11 

3.62 

.14 

Other 

3.70 

.15 

3.79 

.22 

3.68 

.15 

3.66 

.17 

ANOVA Results 

F(2,149) 

= 0.39 

F(2,148) 

= 1.00 

F(2,149) 

= 0.26 

F(2,146) 

= 0.39 

Rank 

0-1 

3.69 

.14 

3.80 

.27 

3.67 

.15 

3.64 

.15 

0-2 

3.65 

.15 

3.70 

.27 

3.64 

.16 

3.63 

.17 

0-3 

3.72 

.16 

3.72 

.35 

3.74 

.18 

3.72 

.16 

0-4 

3.81 

.15 

3.67 

.35 

3.84 

.13 

3.83 

.14 

ANOVA Results 

F(3,148) = 

= 2.81* 

F(3,147) 

= 0.55 

F(3,148) = 

= 5.10* 

F(3,145) 

= 4.5* 

Undergraduate Institution 

USAFA/USNA 

3.64 

.16 

3.69 

.32 

3.65 

.17 

3.62 

.17 

Other Institutions 

3.70 

.15 

3.73 

.30 

3.71 

.17 

3.69 

.17 

ANOVA Results 

F(l,151) = 

= 4.31* 

F(l,149) 

= 0.69 

F(l,151) 

= 3.25 

F(l,147) = 

= 5.50* 


Note . N varies according to missing data and group size. * p < .05 
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In Table 4.3, differences in the mean values of the admission requirements were 
investigated according to gender, race, rank, and undergraduate institution (‘ 1 ’ if from 
US Service Academy, ‘0’ is otherwise) using one-way ANOVA. 

When one-way ANOVA was performed to investigate the significance of 
differences in admission requirement mean values according to gender, no significant 
differences were found for any of the four admission requirements. 

When one-way ANOVA was performed to investigate the significance of 
differences in mean admission requirement values according to race, no significant 
differences were found for any of the four admission requirements. 

When one-way ANOVA was performed to investigate the significance of 
differences in the admission requirement mean values according to rank, significant 
differences were found between the admission requirement means for second lieutenants 
and captains for UGPA (AM = 0.26, p < .05). Significant difference were found between 
second lieutenants and the other three, higher ranks for GRE (verbal) - between 0-1 and 
0-2 (AM = -58.44, p < .05), between 0-1 and 0-3 (AM = -67.90, p < .05), and between 0-1 
and 0-4 (AM = -119.61, p < .05). 

When one-way ANOVA was performed to investigate the significance of 
differences in the admission requirement mean values according to whether or not the 
student attended a US Service Academy (USAFA: N = 47, USNA: N = 2), it was found 
that Service Academy graduate significantly had lower UGPA (AM = -0.13, p < .05), 
GRE (verbal) (AM = -48.60, p < .05), and GRE (quantitative) (AM = -23.58, p < .05) than 
student who attended other undergraduate institutions. 
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Table 4.3 


One-Way ANOVA Results for Demographic Variables (Gender. Race. Rank, and 
Undergraduate Institution) Against Admission Requirements 


Dependent Variables 


UGPA 

GRE (verbaH 

GRE (quantitative) 

UMGPA 

M SD 

M SD 

M m 

M ^ 


Gender 

Male 

3.03 

.31 

523.40 

77.99 

691.84 

56.08 

2.91 

.49 

Female 

2.95 

.27 

535.45 

68.99 

672.73 

50.44 

2.87 

.54 

ANOVA Results 

F(l,161) 

= 1.40 

F(l,161) 

= 0.47 

F(U161) 

= 2.27 

F(l,157) = 

.125 

Race 

White 

3.04 

.30 

527.20 

77.33 

690.70 

54.84 

2.91 

.51 

Black 

2.80 

.28 

513.75 

64.13 

652.50 

50.64 

2.76 

.36 

Other 

2.99 

.22 

506.67 

80.15 

696.67 

63.15 

2.93 

.40 

ANOVA Results 

F(2,160) 

= 2.37 

F(2,160) 

= 0.48 

F(2,160) 

= 1.93 

F(2,156) = 

0.38 

Rank 

0-1 

3.21 

.24 

467.89 

77.72 

696.32 

57.27 

3.05 

.39 

0-2 

3.05 

.29 

526.34 

76.09 

681.41 

58.24 

2.96 

.51 

0-3 

2.94 

.30 

535.80 

71.01 

696.38 

52.69 

2.81 

.50 

0-4 

2.95 

.43 

587.50 

62.38 

672.50 

42.72 

2.94 

.71 

ANOVA Results 

F(3,159) = 

= 4.59* 

F(3,159) = 

= 5.23* 

F(3,159) 

= 1.07 

F(3,155) = 

1.74 

Undergraduate Institution 

USAFA/USNA 

3.06 

.30 

489.55 

75.51 

672.05 

49.44 

2.84 

.38 

Other Institutions 

2.93 

.28 

538.15 

73.25 

695.63 

56.58 

2.93 

.38 

ANOVA Results 

F(l,162) = 

= 6.34* 

F(l,161) = 

^ 13.91* 

F(l,161) = 

= 5.96* 

F(l,157) = 

1.16 


Note . N varies according to missing data and group size. * p < .05 
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Hierarchical Multivariable Regression Results 

While descriptive statistics, alone, provide some insight into the predictive 
validity of the AFIT GEEM admission requirements, hierarchical multivariable 
regression analysis allows for further exploration of the relationships between admission 
requirements and measures of graduate school success, while controlling for potentially 
influential demographic variables - i.e., age, gender and TDELTA. Table 4.2 presents 
the standardized |3 coefficients for each of the five regression models under each 
dependent variable - GGPA, coursework GGPA, thesis grade, and FYGGPA. For all 
four dependent variables, the control variables - age, gender, and TDELTA - were not 
significant (p > .05) predictors of the respective measures of graduate school success 
when entered into a model by themselves. 

For the GGPA model, TDELTA = 0.38, p < .05) and UGPA (g = 0.29, p < .05) 
were both significant only when UGPA was entered into the model beyond the control 
variables. Entering UGPA into the model significantly explained more variance in 
GGPA (AR^ = .07, p < .05), producing an ^ = . 10. When both GRE (verbal) (^ = 0.19, p 
< .05) and GRE (quantitative) (§ = 0.08, p < .05) were entered beyond the controls, only 
GRE (verbal) was significant. Entering both GRE scores beyond the control variables 
significantly explained more variance in GGPA (AR = .05, p < .05), producing an R = 
.08. Only TDELTA (^ = 0.32, p < .05) and UMGPA (§ = 0.35, p < .05) were significant 
when all the independent and control variables were entered into the GGPA model. 
Entering all control and independent variables into the model significantly explained 
more variance in GGPA (AR = .11, p < .05), producing an R = .15. 
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For the coursework GGPA model, only UGPA was signifieant = .21, g < .05) 
in the model that eonsisted of eontrols and only UGPA. Adding UGPA to the model 
signifieantly explained more varianee in eoursework GGPA (AR = .03, p < .05), 
producing an = .10. Adding both the GRE (verbal) and GRE (quantitative) 
significantly explained more variance in coursework GGPA (AR = .08, p < .05), 
producing an ^ = .14. Flowever, only GRE (quantitative) (^ = .17, p < .05) was 
significant beyond the control variables. UMGPA was not significant (^ = .02, p > .05) 
when entered by itself beyond the control variables. When all control and predictor 
variables were entered into the model, none of the predictors were significant. Flowever, 

adding all of the predictors to the model did significantly explain more variance in 

2 2 

coursework GGPA (AR = .10, p < .05), producing an R = .17. 

For the thesis grade model, no admission requirements were significant when 
added individually beyond the control variables. For these models where admission 

2 

requirements were entered individually beyond the control variables, none of the AR 
values were significant (p < .05). When all of the control and independent variables were 
entered into the model, only UMGPA was significant (§ = 0.24, p < .05). Flowever, this 
result was not accompanied by a significant increase in explained variance in thesis grade 
(AR!=-03 ,p<.05). 

The FYGGPA model showed similar results to the GGPA model. The controls 
were not significant predictors of FYGGPA by themselves. Flowever, when UGPA was 
added to the model, both UGPA (^ = 0.29, p < .05) and TDELTA (^ = 0.40, p < .05) 
were significant. Entering UGPA to the model significantly explained an additional 
seven percent of variance in FYGGPA beyond the control variables (AR = .07, p < .05), 
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producing an = .13. When both GRE (verbal) (fi = 0.18, p < .05) and GRE 
(quantitative) (^ = 0.14, p < .05) were entered beyond the eontrols, only GRE (verbal) 
was signifieant. Entering both GRE seores into the model signifieantly explained an 
additional seven pereent of the varianee in FYGGPA beyond the eontrol variables (AR = 
.07, p < .05), producing an R^ = .13. Only TDELTA (^ = 0.23) and UMGPA (§ = 0.05) 
were signifieant (p < .05) when all the eontrol and independent variables were entered 
into the model. Entering all the eontrol and independent variables into the model 
signifieantly explained an additional twelve pereent of the varianee in FYGGPA beyond 
the model with only eontrol variables (AR = .12, p < .05), producing an R = .19. 
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Standardized Beta Coefficients and Regression Goodness of Fit Statistics for AFIT GEEM Admission Requirements 
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Bivariate and Regression Analysis with GRE (Analytical) Scores 

Data was collected on the AFIT GEEM student’s GRE (analytical) score (N = 
163). The mean score of the GRE (analytical) component was M = 637.25 (SD = 74.86). 
The AFIT GEEM program does not require a minimum GRE (analytical) score. A 
ninety-five percent confidence interval about the mean GRE (analytical) score ranges 
from 513.73 to 760.77. Table 4.1 presents the bivariate correlations between variables in 
this study. Of the three measures of graduate school success, the GRE (analytical) 
section significantly correlated with both FYGGPA (r = .17, p < .05) and coursework 
GGPA (r = .20, p < .05). Table 4.3 presents the standardized |3 coefficients for each of 
three regression models - controls alone, controls with GRE (analytical), and controls 
with all predictor variables, including GRE (analytical) for each of four dependent 
variables. 

When regressed against FYGGPA, the GRE (analytical) score was significant as 
the only predictor entered beyond the control variables (^ = 0.17, p < .05). Adding only 
the GRE (analytical) score beyond the controls significantly explained more variance in 
FYGGPA (AR^ = .03, p < .05). When regressed against coursework GGPA, the GRE 
(analytical) score was significant as the only predictor beyond the control variables (^ = 
0.22, p < .05). Adding only the GRE (analytical) score beyond the controls significantly 
explained more variance in coursework GGPA (AR = .05, p < .05). The GRE 
(analytical) score was not significant as the only predictor beyond the controls in either 
the GGPA or thesis grade models. 

When GRE (analytical) was entered into the FYGGPA regression model along 
with the controls and all other independent variables, only UGPA was significant (^ = 
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0.32, E < .05). When all the control and independent variables were entered into the 
FYGGPA model, more variance in FYGGPA was significantly explained (AR = .13, p < 
.05). UGPA was the only significant predictor when all control and independent 
variables were entered into the GGPA = .35, p < .05) and the thesis grade = .24, p < 
.05). The GGPA model that included all the control and independent variables 
significantly explained more variance in GGPA (AR = . 11,P< .05). The thesis grade 
model that included all the control and independent variables did not significantly explain 
more variance in thesis grade (AR = .03, p > .05). When all the control and independent 
variables were entered into the coursework GGPA model, more variance in coursework 
GGPA was significantly explained (AR = . 11,p< .05), although no single predictor was 
significant. 
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Standardized Beta Coefficients and Regression Goodness of Fit Statistics for AFIT GEEM Admission Requirements and GRE 
(AnalvticaO Scores 
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Discussion 


General Conclusion 

This study examined the predictive validity of the AFIT GEEM admission 
requirements - UGPA, GRE (verbal), GRE (quantitative), and UMGPA - against four 
measures of graduate school success - GGPA, thesis grade, coursework GGPA, and 
FYGGPA - for the population of 173 students from the classes of 1995 through 2002. 

Descriptive statistics of the admission requirements and the measures of graduate 
performance revealed that students were admitted from a broad range of undergraduate 
scores, both above and below the stated minimum admission standards, but on the 
graduate performance grade scale, which extended from 3.00 to 4.00, student 
performance huddled within a remarkably tight range. For example, the lower end of the 
ninety-five percent confidence intervals about the mean UGPA (M = 3.02, ^ = .30) and 
the mean UMGPA (M = 2.91, SD = .50) extended from 2.53 and 2.09, respectively. 
Lower-end confidence intervals about the mean GRE (verbal) score (M = 525.03,^ = 
76.75) and the mean GRE (quantitative) score (M = 689.26, ^ = 55.60) extended from 
389.39 and 597.52, respectively. Yet, the confidence interval about the mean GGPA (M 
= 3.69, SD = .16) was distinctively tighter than those about the means of the admission 
requirements, ranging from 3.43 to 3.95. The FYGGPA and coursework GGPA also saw 
a clustering of student performance in a tight range in the upper-half of the graduate 
performance measurement scale. A slightly larger confidence interval extended beyond 
the mean thesis grade (M = 3.72, ^ = .31), ranging from 3.21 to 4.00. The interval was 
wider than those around the other three measures of performance only because thesis 
grades were awarded at distinct levels on the scale of assignable grades. 
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One-way ANOVA of the admission requirements and measures of graduate 
performanee revealed statistieally signifieant differenees in only a few of the mean values 
aeeording to gender, raee, rank, and undergraduate institution (eoded as a ‘ 1 ’ if the 
student attended a US Serviee Aeademy, ‘0’ if otherwise). For example, one-way 
ANOVA did not highlight any statistieally signifieant differenees in the means of any of 
the four admission requirements or any of the four measures of graduate sehool sueeess 
aeeording to raee and gender. This result is quite possibly the product of the small 
number of women (N = 23) and minorities (N = 19) in the population used in this study. 
The population was predominantly white and male and there were, arguably, 
proportionately not enough women and minorities to highlight the differences in 
admission requirements or performance means significantly. This hypothesis is 
supported by the results of the one-way analyses for rank and undergraduate institution, 
where students at different ranks existed in more proportional numbers throughout the 
population. When analyzed by military rank, significant differences in the means of 
UGPA, GRE (verbal), GGPA, coursework GGPA, and FYGGPA appeared. Statistically 
significant differences also appeared for UGPA, GRE (verbal), GRE (quantitative), 
GGPA, and FYGGPA according to whether or not the student attended a US Service 
Academy (coded as a ‘1’). 

Bivariate correlation analysis revealed weak to moderate statistically significant 
relationships between the admission requirements and the measures of graduate school 
success. In general, the UGPA, GRE (verbal), and GRE (quantitative) score correlated 
significantly with the GGPA, coursework GGPA, and FYGGPA. Although not strong, 
there was some weak-to-moderate relationship between the admission requirements and 
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the graduate grade-based measures of performanee. UMGPA, however, demonstrated no 
signifieant eorrelations with any of the measures of graduate sehool performanee. If the 
purpose of the AFIT GEEM admission proeess is to identify those students with the 
greatest potential to perform, as defined graduate grades and the thesis grade, then the 
UMGPA is a useless predietor. It signifieantly explains no varianee in the dependent 
measures of graduate school performance and should not be used as a discriminator 
among applicants. The thesis grade did not correlate significantly with any of the AFIT 
GEEM admission requirements. The admission requirements have no predictive validity 
against the thesis grade and, therefore, cannot differentiate between applicant levels of 
graduate performance potential. 

Flierarchical regression analysis revealed that the admission requirements are 
more often significant predictors of graduate school success when entered into a 
regression model by themselves. When entered into a regression model together, there is 
an increase in the variance explained - most likely due to an increase in the number of 
variables in the model - but few, if any, of the predictors are significant. For example, 
when UGPA was entered by itself, it was a significant predictor for GGPA = .29, p < 
.05), coursework GGPA (^ = .21, p < .05), and FYGGPA (§ = .29, p < .05). Significant 
coefficients for UGPA were also found for GGPA = .35, p < .05), thesis grade (§ = 

.24, p < .05), and FYGGPA = .32, p < .05) when all controls and independent 
variables, including GRE (analytical) were entered into the regression model. Significant 
coefficients were derived for GRE (verbal), entered by itself beyond the control variables, 
for GGPA (^ = .19, p < .05) and FYGGPA (^ = .18, p < .05). A significant coefficient 
for GRE (quantitative), entered by itself beyond the control variables, was derived only 
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against coursework GGPA (^ = . 17, 2 < -05). Significant coefficients for UMGPA were 
only derived in eombination with all eontrols and other independent variables against 
eoursework GGPA = -17, p < .05). The eoeffieients for GRE (analytieal), entered by 
itself beyond the eontrol variables, were only signifieant for eoursework GGPA (§ = .22, 
P < .05) and FYGGPA (^ = • 17, p < .05). TDELTA was the only eontrol variable that 
had signifieant Beta eoeffieients: with UGPA against GGPA = .38, p < .05) and 
FYGGPA (§ = .40, p < .05) and in eombination with all other independent variables for 
GGPA (§ = .32, p < .05), FYGGPA without GRE (analytieal) = .32, p < .05), and 
FYGGPA with GRE (analytieal) (^ = .31, p < .05). 

The deseriptive, ANOVA, eorrelation, and regression results of this study support 
the eonelusion that the AFIT GEEM admission requirements are not the only influenees 
on the student’s performanee in graduate sehool. There must be some other influenees 
that eause the elustering of student graduate performanee despite broad ranging 
undergraduate baekgrounds. One-way ANOVA shows that military rank and 
undergraduate institution may have some influenee on performanee, for the means of 
graduate performanee variables signifieantly differed by these elassifieations. Although 
the ANOVA did not show any signifieant differenees in means aeeording to raee and 
gender, these findings are suspeet eonsidering the small representation of these groups in 
the population. So, while the one-way analysis on raee and gender did not direetly 
provide support for the eonelusion that there are other influenees on graduate sehool 
performanee, it did not direetly work against the eonelusion either. Weak-to-moderate 
eorrelations between admission requirements and measures of graduate sehool suceess 
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further support the eonelusion that there are other influenees on graduate sehool 
performanee than the stated admission requirements. 

Currently, UGPA, GRE (verbal), GRE (quantitative), and UMGPA are, offieially, 
the only influenees on the student’s admission to the AFIT GEEM program. The AFIT 
GEEM program does not offieially eonsider gender, raee, rank, or undergraduate 
institution in admitting students. The results of this study do not imply that the AFIT 
GEEM admission proeess is neeessarily a bad proeess. These results merely suggest that 
the eurrent AFIT GEEM admission is simply not the best proeess and that the proeess 
may be improved by looking at other influenees on graduate sehool performanee. 
Assumptions 

The major eonelusion of this study - that there are other influenees on graduate 
performanee - relies upon two assumptions: first, generalizations eould be made from the 
population used; and seeond, the measures of graduate sehool sueeess used in this study 
were appropriate to the goals of the AFIT GEEM program. 

Population. Before performing any statistieal analysis, it was thought that the 
tremendous homogeneity found in the population would produce higher predictive 
validity coefficients than those published in most other studies. It was thought that racial 
homogeneity (most of the population was White) would serve to minimize concerns over 
racial bias in the UGPA and GRE General Test scores. Gender homogeneity (most of the 
population was male) would serve to minimize concerns over gender bias in the UGPA or 
GRE General Test scores. Considering that the majority of students in the population 
were US military officers with engineering undergraduate degrees and that a good 
number of the students (47 of the 173 students) came from the same undergraduate 
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institution - the US Air Force Academy, it was thought that this study would succeed in 
minimizing the “noisy data problems” caused by differing backgrounds and academic 
experience (Schneider and Briel, 1990). 

Flowever, the results of this study did not produce the expected predictive validity 
coefficients. Bivariate correlations of the admission requirements against FYGGPA were 
slightly lower than those published in the ETS’s 2000-2001 Guide to the Use of Scores 
for engineering graduate students. Flomogeneity of the population cannot be blamed, 
specifically, for this study’s slightly weaker relationships. The ETS qualifies its results 
with a warning that “available samples of minority students.. .have been very small” 

(ETS Guide to the Use of Scores. 2000-2001: p. #9). Rather, homogeneity can, more 
appropriately, be suspected for contributing to the weak predictive relationships, in 
general. For example, while most of the population was White, not all of the population 
was White. While most of the population was male, not all of the population was male. 

If racial and gender bias existed in UGPA and GRE measures recorded in this study, it is 
possible that those biases were not adequately accounted for in the correlation and 
regression coefficients. When the overwhelming majority of the population bears a 
specific characteristic - i.e., eighty-seven percent of the population was male — the 
contribution of the minority characteristics are minimized in correlation and regression 
analyses. If the contributions of these students are minimized, then this study cannot 
completely investigate the predictive validity of the AFIT GEEM admission 
requirements. 

If homogeneity of the population does, in fact, reduce the predictive validity of 
the coefficients, then questions about the generalizability of this study’s results are 
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appropriately raised. Originally, this study was designed to foeus, speeifieally, on 
validating the AFIT GEEM admission eriteria. Generalizability beyond the eontext of 
the GEEM program was not a primary eoncem at the outset. However, in hindsight, the 
results of this study may be of interest to other eighty-one graduate engineering 
management programs published in the 2001 Peterson’s Guide to Graduate Education in 
Engineering and the Applied Sciences . The AFIT GEEM program admission 
requirements and process are not terribly different from the majority of these programs. 
Other validity studies on graduate engineering management admission requirements may 
benefit from the realization that the UMGPA does not correlate with any of the four 
measures of graduate school success. They may benefit from the realization that the 
thesis grade does not satisfy the normality and constant error variance assumptions of the 
General Linear Model (GEM) and that none of the admission requirements significantly 
correlated with the thesis grade. Most importantly, other studies may benefit from the 
realization that although AFIT admits students both well below and above its minimum 
admission requirements, the mean AFIT GEEM GGPA is M = 3.69 (SD = .16) with a 
ninety-five percent confidence interval about that mean ranging from 3.43 to 3.95. Other 
programs may find that their admission requirements perform in a manner similar to 
those of the AFIT GEEM program and can enhance their own admissions processes by 
identifying the other influences on graduate performance. 

Measures of Graduate School Success. Before examining the validity of 
admission requirements or positing any other influences on graduate performance, 
however, the conceptualization of graduate school success must developed. The 
measures of graduate school success must be in-line with the goals of the respective 
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graduate program. The AFIT GEEM program foeuses on eoursework and researeh, 
aceomplished through the thesis study. As seen in this study, however, the thesis grade 
was statistically not a good measure of graduate performance. The thesis grade does not 
satisfy two of the major assumptions of the General Linear Model and, thereby, prevents 
the use of regression analysis. 

However, there were not too many alternative definitions for graduate school 
success in the AFIT GEEM program. For the classes of 1995 through 2002, 99.4% of 
students graduated within eighteen month and completed a thesis study. There has been 
no consistent capstone course in the AFIT GEEM program over the study period. 
Graduate grade-based measures of success were the only viable alternatives for the AFIT 
GEEM program. Dependent measures like the FYGGPA and eoursework GGPA 
captured student performance in the graduate classroom but neglected the research aspect 
of the AFIT experience. Perhaps the best measure, from a definitional standpoint, was 
the GGPA, which captured the six quarters of eoursework grades and the thesis grade. 
Despite the GGPA being the optimal conceptualization - strictly in the sense that in 
captured both the eoursework and research aspects of the AFIT GEEM program’s focus - 
the admission requirements still did not demonstrate tremendous predictive validity of 
this measure. 

On that point, this study recommends one of two courses of action: either keep the 
GGPA as the measure of success and search for other influential admissions data; or find 
a different measure of graduate school success for which the current admission 
requirements demonstrate a higher predictive validity. 
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Appendix A 


Number and Types of Degrees Earned 
by AFIT GEEM Students from 1995- 
2002 

Degree 

Count 

AeroEng 

1 

Arch Eng 

1 

BAE, Arch Engr 

1 

BS Architecture 

1 

BS ENG SCI 

1 

BS Physics 

1 

BS, Aero Engr 

1 

BS, Architecture 

2 

BS, Biology 

2 

BS, CE 

79 

BS, CE (ENV) 

8 

BS, CE (HCON) 

1 

BS, Chemistry 

1 

BS, Chem Engr 

1 

BS, EE 

24 

BS, EE (BEE) 

4 

BS, ENG 

1 

BS, Economics 

1 

BS, Engr Mech 

1 

BS, Env Eng 

3 

BS, Env Sci 

1 

BS, Geo/Math 

1 

BS, Geology 

1 

BS, Industrial Engineering 

6 

BS, Inti Affairs 

1 

BS, Mech Eng 

27 

BS, Oceanography 

1 

BS, OE 

1 

Total 

174 


*One student obtained two BS degrees: a BS, 
EE from SUNY Binghamton and a BS, 
Biology from USAFA. 
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Appendix B 


Number of Students in AFIT GEEM (1995-2002) for Varied Undergraduate Institutions 

Undegraduate Institution 

Count 

Undergraduate Institution 

Count 

Arizona SU 

2 



Auburn 

2 

Stevens Institute 

1 

BYU 

2 

Syracuse U 

2 

Carnegie Mellon 

2 

Texas A&M 

5 

Citadel 

1 

Texas Tech 

1 

Clarkson U 

3 

Tufts 

1 

Clemson 

3 

U Akron 

1 

FI Inti U 

1 

U Alabama 

1 

Florida SU 

1 

U Arizona 

1 

Geneva College 

1 

U Cinncinnati 

2 

Georgia Tech 

4 

U Florida 

2 

Flumbolt State 

1 

U Illinois 

3 

Illinois IT 

1 

U Kentucky 

2 

Illinois SU 

1 

U Louisville 

2 

LSU 

2 

U Maine 

1 

Lehigh 

2 

U Minnesota 

2 

Loyola Marymont 

1 

U Missouri 

1 

MIT 

1 

U Missouri - Columbia 

1 

Michigan SU 

1 

U Missouri - Rolla 

3 

Milwaukee Sc Eng 

1 

U New Flampshire 

3 

Montana SU 

3 

U Oklahoma 

1 

NC A&T 

1 

U Portland 

2 

NC State 

1 

U Santa Clara 

2 

New Mexico SU 

1 

U So Florida 

1 

Notre Dame 

1 

U Tennessee 

2 

OK Christian U 

2 

U Texas 

1 

Ohio SU 

4 

U Toledo 

1 

OhioU 

1 

U Washington 

2 

Oklahoma SU 

1 

USAFA 

47 

Oregon SU 

3 

use 

1 

Penn State 

5 

USNA 

2 

Purdue 

1 

UVA 

1 

RPI 

3 

VMI 

3 

Rose-Hulman 

2 

Vanderbilt 

1 

SUNY Binghamton 

1 

W. New Eng. College 

2 

So Dakota School of Mines 

1 

Washington SU 

1 

So Illinois U 

1 

Worcester PI 

1 

South Dakota SU 

2 

Total 

173 
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Appendix C. 1 


10 October 2C)Q1 


MEMORANDUM FOR AFIT/RRD 

FROM; AFTT/ENV 

SUBJECT: Request for Access lo Educational Records for Research 

1. In partial fulfillment of the AETT graduation lequitrmerts^ 1 have proposed to conduct a 
thesis study, focused on eAamining the validity of graduate level admissions criteria as 
predictors of academic success for students in the Engrncenng and Environmental 
Management program. The proposed study relics upon information about current students 
and graduates found in education records and the personnel data system (PDS). This letter 
documents a formal request before AFIT/RRD for access to educatiwial records and the PDS 
in pursuit of the proposed study, 

2. Based on 34 CFR § 99,31 <aXl> of the Family Educational Rights and Privacy Act of 1974 
IFERPA) regulation, disclosure of educational records is allowed, within an agency, when 
the agency has determined the requestor has a ^legitimate educational interest.” I submit to 
you ^at the proposed study meets such a standard for it seeks to advance the Institute’s 
ability, in the context of the contemporary applicant pooh to select those candidates most 
likely to thrive m the AFIT academic environment. The publication of several preceding, 
similarly focused, theses efforts testifies to continual need for reexamination of such a study 
with the progressive changes in the Air Force's demographic. In light of programmatic 
changes and several subsequent years of continuity. I maintain that the time has come to, 
againr reexamine the admissions criteria for graduate students in the Engineering and 
Environmental Management program. Additional guidance on this issue can be found in the 
1998 “Guidelines for Postsecondary Institutions for ImpLemcntation of the Family 
Educational Rights and Privacy Act of 1974 as Amended ” published by the American 
Association of Collegiate Registrars and Admissions Officers. The example provided or 
page 72 justifies granting a similar requesL 

3* Any and all information obtained as part of the proposed study will remain confidential 
among Major Mark Ward, Dr. Charles Bleckmann, ^jor Daniel Holt, and myself. No 
infarmation about any individual will be disclosed to anyone beyond those persons 
previously named. Results of this research will be presented only in aggregate form. Names 
of the data subjects will appear nowhere in the study or any associated reports. 

4. The proposed study considers all students enrolled in AFTT^s Graduate Engineering and 
Environmenial Management prograni from 1993 through 2001 to be viable data subjects. 
The proposed predictors of academic success mirror the AFTT admission requirements: 
Undergraduate Grade Point Average (UGPA) and Graduate Review Examination (GRB) 
and/cHT Graduate Management Admission Test (GMAT) scores. The proposed measure of 
academic success relies upon Cumulative and Quarterly AFTT GPA and AFIT thesis grade, 
as documented in educational records. To the best of my knowledge, the data hereby 
requested is sufficient to execute the proposed study. However, should different data be 
required, I will submit an second request to AFTT/RRD at that time. 
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Appendix C.2 


5. PleAse flirect all quesUons lo cither me or Major Daniel Holi. i can be reached at (937)431 
8565 or charlcs.zitzmann@afit.edu . Major Holt can be reached at X 2998 or 
dan i e I .holt @ afit.edu . 


CHARLES C, zrrZMAN, LT, U$AF 

Graduate Student 

AFTT/ENV 



AFTT/ENV 
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Appendix C.3 


!S 


Registrar’s Office 

Air Force Institute of Technology 

Admis-siunt tind Dli^i^Ctvratc, Wrlght-P.at(j^JWn Air Porcc ftlH, Olll4k 45433*^1102 

Memorandum for Record 15 Oct 2001 

Access lo Student Records 

]. Purpose of the research; Determine the predictive validity of the Engineering & 
Environmental Management's selection criteria. That is, do the selection criteria predict 
graduate school success. 

2. Data needed (e.g., GPA, GMAT, GRE): Undergraduate GPA, Undergraduate math . 
GPA, GRE scores, AFiT GPA, AFIT Thesis grade 

3. The group to which the needed data is limited (e.g., speciEc progianv GPA < 3.0, etc): 
Studeiils ejirolled and graduated from the GEE program since 1993. 


Signature (I agree p,p^teci the confidentiality of the data g^ercdl (dale) 

CHARLES C Z1T™ANN, IsI Lt. USAF 


fly v/Ji _™ 

iJtstrucLor/Advisor's signature (1 approve the project described abovcMeeM (Ste) 

DANIEL T. HOLT, Major, USAF 


/ 


tNDELL BAKER 
Associate Registrar 
Air Force Institute of I'ectioolog) 
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Figure 1 

Frequency Fiistogram of GGPA 


Appendix D.l 
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Figure 2 

Frequency Fiistogram of Coursework GGPA 
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Appendix D.2 

Figure 3 

Frequency Fiistogram of Thesis Grade 



Figure 4 

Frequency Fiistogram of FYGGPA 



60 








Bibliography 

Abedi, J. (1991). Predicting Graduate Academic Success from Undergraduate Academic 
Performance: A Canonical Correlation Study. Educational and Psychological 
Measurement, 31, 151-160. 

Air Force Institute of Technology. Air Force Institute of Technology, Graduate School of 
Engineering and Management, Academic Year 2001-2002 Catalog. Author. 

Ahmadi, M., Raiszadeh, F., & Flelms, M. (1997). An examination of the admission 
criteria for MBA programs: A case study. Education, 117(4), 540-546. 

Betts, J.R., & Morell, D. (1998). The determinants of undergraduate grade point average 
— the relative importance of family, background, high school resources, and peer 
group effects. The Journal of Human Resources, 34(2), 268-293. 

Braun, FI.l., & Jones, D.FI. (1985). Use of empirical Bayes methods in the study of 

validity of academic predictors of graduate school performance (ETS Research 
Report 84-34). Princeton. NJ: Educational Testing Service. 

Burton, N., & Turner, N. (1983). Effectiveness of Graduate Record Examinations for 
predicting first year grades: 1981-82 summary report of Graduate Record 
Examinations Validity Study Service. Princeton, NJ: Educational Testing 
Service. 

Case, D.O., & Richardson, J.V. (1990). Predictors of student performance with 

emphasis on gender and ethnic determinants. Journal of Education for Library 
and Information Science, 30, 163-182. 

Dunlap, W.R. (1979). Flow effective are graduate social work admission criteria? 
Journal of Education for Social Work, 15, 96-102. 

Enright, M.K., & Gitomer, D. (1989). Toward a description of successful graduate 


61 



students (GRE Board Research Report 85-17R). Princeton, NJ: Educational 
Testing Service. 

ETS. 2000-2001 ETS Guide to the Use of Scores. Princeton, NJ. 

Goldberg, E.L., & Alliger, G.M. (1992). Assessing the validity of the GRE for students 
in psychology: A validity generalization approach. Educational and 
Psychological Measurement, 52, 1019-1027. 

Goldman, R.D., & Slaughter, R.E. (1976). Why college grade point average is difficult 
to predict. Journal of Educational Psychology, 68, 9-14. 

Gunne, M.G., & Leslie, L.L. (1972). Exceptional graduate admissions at the 

Pennsylvania State University. Pennsylvania State University: Center for the 
Study of Higher Education. (ERIC Document Reproduction Service No. ED 061 
889). 

Hagedom, L.S., & Nora, A. (1996). Rethinking admissions criteria in graduate and 
professional programs. New Directions for Institutional Research, 92, 31-44. 

House, J.D. (1994). Student motivation and achievement in college chemistry. 
InternationalJournal of Instructional Media, 21, 1-11. 

House, J.D. (1989). Age bias in prediction of graduate grade point average from 
Graduate Record Examination scores. Educational and Psychological 
Measurement, 49, 663-666. 

House, J.D., Gupta, S., & Xiao, B. (1997). Gender differences and prediction of grade 

performance from Graduate Record Examination scores and undergraduate grades 
for length of time to completion of degree. Psychological Reports, 71, 1019- 
1022. 


62 



House, J.D., Johnson, J.J., & Tolone, W.L. (1987). Predictive validity of Graduate 
Record Examination score for performance in selected graduate psychology 
courses. Psychology Reports, 60, 107-110. 

Howard, L.C. (1968) Graduate Education for the ‘disadvantaged’and black-oriented 
university graduates. Paper presented at the Council of Graduate Schools, 
Washington, DC. (ERIC Document Reproduction Service No ED 026 022). 

Juola, A. (1977). Grade inflation in higher education; What can or should we do? 

(ERIC Document Reproduction Service No ED 129 917). 

Juola, A. (1980). Grade inflation in higher education - 1979: Is it over? (ERIC 
Document Reproduction Service No ED 189 129). 

Keith, R.W. (1977). Success Predictors For Students to AttendAFITDepartment of 

Systems Management Master’s Degree Programs. Unpublished master’s thesis. 
School of Engineering, Air Force Institute of Technology, Wright-Patterson AFB, 
OH. 

Kuncel, N.R., Hezlett, S.A., & Ones, D.S. (2001) A comprehensive meta-analysis of the 
predictive validity of the Graduate Record Examinations: Implications for 
graduate student selection and performance. A comprehensive meta-analysis of 
the predictive validity of the Graduate Record Examinations: Implications for 
graduate student selection and performance. Psychological Bulletin, 127(1), 162- 
181. 

Levine, A. (1994). To deflate grade inflation: simplify system. Chronicle of Higher 
Education, 40, B3. 

Malone, B.G., Nelson, J.S., & Nelson, C.V. (2001). Admitting at-risk students into a 


63 



principal preparation program: predicting success. American Secondary 
Education, 29(4), 2-17. 

Matthews, T.A., & Martin, D.J. (1992). Reciprocal suppression and interaction effects 
of age with undergraduate grades and GRE on graduate performance in a eollege 
of education. Educational and Psychological Measurement, 28, 453-456. 

McKee, A.J., Mallory, S.L., & Campbell, J. (2001). The graduate record examination 
and undergraduate grade point average: Predicting graduate grade point averages 
in a criminal justice graduate program. Journal of Criminal Justice Education, 
12(2), 311-317. 

Michael, J.J. (1983). The prediction of academic achievement in graduate study in 
education. Educational and Psychological Measurement, 55, 309-316. 

Milner, M., McNeil, J.S., & King, S.W. (1984). The GRE: A question of validity in 

predicting performance in professional sehools of social work. Educational and 
Psychological Measurement, 44, 945-950. 

Morrison, T., & Morrison, M. (1995). A meta-analytic assessment of the predictive 
validity of the Quantitative and Verbal components of the Graduate Record 
Examination with graduate grade point average representing the criterion of 
graduate school success. Educational and Psychological Measurement, 55, 309- 
316. 

Neter, J., Kutner, M.H., Nachtsheim, C.J., and Wasserman, W. Applied Linear Statistical 
Models, ed. McGraw-Hill: 1996. 

Norcross, J.C., Hyanch, J.M., & Terranova, R.D. (1996). Graduate study in psychology: 
1992-1993. American Psychologist, 51, 631-643. 


64 



Payne, D.A., Wells, R.A., & Clarke, R.R. (1971). Another eontribution to estimating 

sueeess in graduate sehool: A seareh for sex differenees and eomparison between 
three degree types. Educational and Psychological Measurement, 31, 497-503. 

Peterson’s. 2002 Graduate Programs in Engineering and Applied Sciences, 36‘^ ed., 
Books, p. 1994-1200. 

Roek, D.A., Werts, C. & Grandy, J. (1982). Construct validity of the GRE aptitude test 
across populations - An empirical confirmatory study (GRE Board Professional 
Report No 78-lP; ETS Researeh Report 81-57). (Prineeton, NJ; Educational 
Testing Service). 

Schneider, L.M., and Briel, J.B. (1990). Validity of the GRE: 1988-89 summary report. 
Princeton, NJ: Educational Testing Service. 

Shelton, P.D. (2001). Admissions tests: not perfect, just the best measures we have. The 
Chronicle of Higher Education, 47(43), B15. 

Singer, J.E. (1964). The use of manipulative strategies: Machiavellianism and 
attractiveness. Sociometry, 27, 128-150. 

Stahmer, H.M. (1968). The disadvantaged student in graduate school. The Harvard- 
Yale-Columbia Intensive Summer Studies Programs. Washington, DC: Council 
of Graduate Schools. (ERIC Document Reproduction Service No. ED 026 020) 

Sternberg, R.J., & Williams, W.M. (1997). Does the Graduate Record Examination 

predict meaningful success in the graduate training of psychologists? American 
Psychologist, 52(6), 639-641. 

Thomell, J.G., & McCoy, A. (1985). The predictive validity of the Graduate Record 


65 



Examination for subgroups of students in different aeademie diseiplines. 
Educational and Psychological Measurment, 45, 415-419. 

Van Seotter, J.R. (1983). Predictors of Success in Air Force Institute of Technology 
Resident Master’s Degree Programs: A Validity Study. Unpublished master’s 
thesis. Sehool of Systems and Logisties, Air Foree Institute of Teehnology, 
Wright-Patterson AFB, OH. 

Wilson, K.M. (1979). The validation of GRE scores as predictors of first-year 

performance in graduate study: Report of he GRE Cooperative Validity Studies 
Project (GRE Board Researeh Report GREB 75-8R). Prineeton, NJ: Educational 
Testing Service. 


66 



Vita 


First Lieutenant Charles C. Zitzmann graduated from Chaminade Fiigh Sehool in 
Mineola, Long Island, New York. Fie entered undergraduate studies at the United States 
Air Foree Aeademy, where he graduated with a Baehelor of Seienee degree in Legal 
Studies in May 1998. 

Fiis first assignment was at Wright-Patterson AFB as a eost analyst in the 
Reeonnaissanee System Program Offiee, Global Flawk Unmanned Aerial Vehiele 
Program in August 1998. In August 2000, he entered the Graduate Sehool of 
Engineering and Management, Air Foree Institute of Teehnology. Upon graduation, he 
will be assigned to the Air Foree Cost Analysis Ageney in Crystal City, Virginia. 


67 



REPORT DOCUMENTATION PAGE 

Form Approved 

0MB No. 074-0188 

The public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and 
maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of the collection of information, including 
suggestions for reducing this burden to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and Reports (0704-0188), 1215 Jefferson Davis Highway, 

Suite 1204, Arlington, VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to an penalty for failing to comply with a collection of 
information if it does not display a currently valid 0MB control number. 

PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS. 

1. REPORT DATE (DD-MM-YYYY) 
26-03-2002 

2. REPORT TYPE 

Master’s Thesis 

3. DATES COVERED (From - To) 

Aug 2001 - Mar 2002 


4. TITLE AND SUBTITLE 

THE PREDICTIVE VALIDITY OF THE AFIT GRADUATE ENGINEERING 

AND ENVIRONMENTAL (GEEM) ADMISSION REQUIREMENTS. 

5a. CONTRACT NUMBER 

5b. GRANT NUMBER 

5c. PROGRAM ELEMENT NUMBER 

6. AUTHOR(S) 

Zitzmann, Charles, C., First Lieutenant, USAF 

5d. PROJECT NUMBER 

5e. TASK NUMBER 

5f. WORK UNIT NUMBER 

7. PERFORMING ORGANIZATION NAMES(S) AND ADDRESS(S) 

Air Force Institute of Technology 

Graduate School of Engineering and Management (AFIT/EN) 

2950 P Street, Building 640 

WPAFB OH 45433-7765 

8. PERFORMING ORGANIZATION 

REPORT NUMBER 

AFIT/GAQ/ENV/02M-16 

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 

AFIT ENV 

ATTN: Daniel T. Holt, Maj, USAF (937) 255-3636 X4574 

Air Force Institute of Technology e-mail: daniel.holt@afit.edu 

Graduate School of Engineering and Management (AFIT/EN) 

2950 P Street, Building 640 

WPAFB, OH 45433-7765 

10. SPONSOR/MONITOR’S ACRONYM(S) 

11. SPONSOR/MONITOR’S REPORT 
NUMBER(S) 

12. DISTRIBUTION/AVAILABILITY STATEMENT 

APPROVED FOR PUBLIC RELEASE; DISTRIBUTION UNLIMITED. 

13. SUPPLEMENTARY NOTES 

14. ABSTRACT 

Recognizing that a “strong commitment to advanced technical and management education” (Van Scotter, 1993, p. 1) serves 
to maintain the capability and strength of the United States Air Force (USAF), the Air Force Institute of Technology (AFIT) Graduate 
School of Engineering and Management (GSEM) offers a full-time, eighteen-month. Graduate Engineering and Environmental 
Management (GEEM) program to military officers and Department of Defense (DoD) civilian employees. The AFIT uses the 


applicant’s undergraduate grade-point average (UGPA), Graduate Record Examination (GRE) General Test verbal and quantitative 
scores, and undergraduate mathematics course grade-point average (UMGPA) to select applicants for admission. 

This study analyzed how well these selection devices predicted success in the GEEM program for students admitted to the 
classes of 1995 through 2002. Controlling for age, gender, and the time (in years) between undergraduate and graduate education 
(TDELTA), the UGPA, GRE (verbal) score, GRE (quantitative) score, and UMGPA were sequentially entered into a hierarchical, 
multivariable, linear regression model. Using this technique, three separate regression model were built, one for each 
operationalization of graduate school performance: cumulative graduate grade-point average (GGPA), AFIT thesis grade, and first- 
year graduate grade-point average (FYGGPA). 

Every independent variable, either alone or in combination with other predictor variables, except for age, gender, and GRE 
(quantitative), demonstrated statistical significance as a predictor of both FYGGPA and GGPA. Only UMGPA, combined with all 
other predictor variables, significantly predicted thesis grade. _ 

15. SUBJECT TERMS 

GRE, Graduate Education, Grade-Point Average, Admission Requirements, Graduate School Success, Academic Performance 


1 16. SECURITY CLASSIFICATION OF: 

17. LIMITATION OF 

18. NUMBER 

19a. NAME OF RESPONSIBLE PERSON 




ABSTRACT 

OF 

Mark A. Ward, Maj, USAF (ENV) 

a. REPORT 

b. ABSTRACT 

c. THIS PAGE 


PAGES 

19b. TELEPHONE NUMBER (Include area code) 

u 

u 

u 

UU 

77 

(937) 255-6565, ext 4314; e-mail: mark.ward(gafltedu 


Standard Form 298 (Rev. 8-98) 


Prescribed by ANSI Std. Z39-18 



































